Synesthesia of Machines Towards Intelligent Multi-modal Sensing-Communication Integration
CHENG Xiang1, ZHANG Haotian1, LI Sijiang1, HUANG Ziwei1, YANG Zonghui1, GAO Shijian2, BAI Lu3,4, ZHANG Jia'nan1, ZHENG Xinhu5, YANG Liuqing5,6,7
1. School of Electronics, Peking University, Beijing 100871; 2. Samsung Semiconductor, Samsung SoC Research and Deve-lopment Lab, San Diego, CA 92121, USA; 3. Shandong Research Institute of Industrial Technology, Jinan 250100; 4. Joint SDU-NTU Centre for Artificial Intelligence Research, Shangdong University, Jinan 250101; 5. Intelligent Transportation Thrust, The Hong Kong University of Science and Technology(Guangzhou), Guangzhou 511455; 6. Internet of Things Thrust, The Hong Kong University of Science and Technology(Guangzhou), Guangzhou 511455; 7. Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong 999077
Abstract Integrated sensing and communications(ISAC) technique is limited to the sharing of radar sensing and communications at the spectrum and hardware levels, and it fails to enhance the performance of communication and sensing in future emerging application scenarios. In scenarios involving massive multi-modal sensing and communication data, ISAC should evolve towards the incorporation of multi-modal sensing, specifically intelligent multi-modal sensing-communication integration. Inspired by human synesthesia, a paradigm for intelligent multi-modal sensing-communication integration, synesthesia of machines(SoM), is systematically established and discussed in this paper. Firstly, three typical operational modes of SoM , SoM-evoke, SoM-enhance and SoM-concert, are systematically summarized, and thus the purposes and methods of the mutual assistance and enhancement between communications and multi-modal sensing are given comprehensively. Then, the data foundation of SoM research, mixed multi-modal sensing and communication(M3SC) simulation dataset, and the theoretical foundation of SoM research, SoM mechanism, are also discussed. Finally, the current research status of SoM is reviewed and future research directions are prospected.
Fund:National Key Research and Development Project of China(No.2020AAA0108101), National Natural Science Foun-dation of China(No.62125101,62341101,62001018,62301011,62373315,U23A20339,62371273), New Cornerstone Science Foundation through the XPLORER PRIZE, Natural Science Foundation of Shandong Province(No.ZR2023YQ058), Young Elite Scientists Sponsorship Program by CAST(No.2023QNRC001), Taishan Scholars Program, Guangzhou Municipal Science and Technology Project(No.2023A03J0011,2023A03J06831), and Guangdong Provincial Department of Education Major Research Project(No.2023ZDZX1037)
Corresponding Authors:CHENG Xiang, Ph.D., professor. His research interests in-clude data-driven intelligent network and net-worked intelligence.
About author:: ZHANG Haotian, Ph.D. candidate. His re-search interests include multi-modal sensing-assisted transceiver design in communication systems.Li Sijiang, Ph.D. candidate. His research interests include cooperative localization, per-ception and intelligence of connected mobile agents.HUANG Ziwei, Ph.D. candidate. His re-search interests include complex high-mobility communication channel measurements and mo-deling.YANG Zonghui, Ph.D. candidate. His re-search interests include wireless communica-tions and integrated sensing and communi-cations.GAO Shijian, Ph.D., senior engineer. His research interests include wireless communica-tions and statistical signal processing.BAI Lu, Ph.D., professor. Her research interests include 6G wireless communication network channel measurements and modeling.ZHANG Jia'nan, Ph.D., assistant profe-ssor. His research interests include network theory and networked intelligence.ZHENG Xinhu, Ph.D., assistant professor. His research interests include multi-modal perception, multi-agent cooperative perception and networked intelligence.YANG Liuqing, Ph.D., professor. Her research interests include wireless communica-tion networks, multi-agent systems and inte-grated communication and sensing.
[1] NAKAMURA T.5G Evolution and 6G. IEEE Symposium on VISI Technology, 2020. DOI: 10.1109/VLSITechnology18217.2020.9265094. [2] CHOWDHURY M Z, SHAHJALAL M, AHMED S, et al. 6G Wire-less Communication Systems: Applications, Requirements, Techno-logies, Challenges, and Research Directions. IEEE Open Journal of the Communications Society, 2020, 1: 957-975. [3] WANG Z Q, DU Y, WEI K J, et al. Vision, Application Scena-rios, and Key Technology Trends for 6G Mobile Communications. Science China Information Sciences, 2022, 65(5). DOI: 10.1007/s11432-021-3351-5 [4] LIU A, HUANG Z, LI M, et al. A Survey on Fundamental Limits of Integrated Sensing and Communication. IEEE Communications Sur-veys and Tutorials, 2022, 24(2): 994-1034. [5] 林粤伟,王溢,张奇勋,等.面向6G的通信感知一体化车联网研究综述.信号处理, 2023, 39(6): 963-974. (LIN Y W, WANG Y, ZHANG Q X, et al. Overview of the Re-search on 6G Oriented Internet of Vehicles for Integrated Sensing and Communication. Journal of Signal Processing, 2023, 39(6): 963-974.) [6] 程翔,张浩天,杨宗辉,等.车联网通信感知一体化研究:现状与发展趋势.通信学报, 2022, 43(8): 188-202. (CHENG X, ZHANG H T, YANG Z H, et al. Integrated Sensing and Communications for Internet of Vehicles:Current Status and Development Trend. Journal on Communications, 2022, 43(8): 188-202.) [7] STURM C, WIESBECK W.Waveform Design and Signal Processing Aspects for Fusion of Wireless Communications and Radar Sensing. Proceedings of the IEEE, 2011, 99(7): 1236-1259. [8] CHENG Z Y, LIAO B, SHI S N, et al. Co-design for Overlaid MIMO Radar and Downlink MISO Communication Systems via Cramér-Rao Bound Minimization. IEEE Transactions on Signal Processing, 2019, 67(24): 6227-6240. [9] LI D, ZHAN M Y, LIU H Q, et al. A Robust Translational Motion Compensation Method for ISAR Imaging Based on Keystone Trans-form and Fractional Fourier Transform under Low SNR Environment. IEEE Transactions on Aerospace and Electronic Systems, 2017, 53(5): 2140-2156. [10] 曾浩,吉利霞,李凤,等.16QAM-LFM雷达通信一体化信号设计.通信学报, 2020, 41(3): 182-189. (ZENG H, JI L X, LI F, et al. 16QAM-LFM Waveform Design for Integrated Radar and Communication. Journal on Communica-tions, 2020, 41(3): 182-189.) [11] 李晓柏,杨瑞娟,陈新永,等.基于分数阶傅里叶变换的雷达通信一体化信号共享研究.信号处理, 2012, 28(4): 487-494. (LI X B, YANG R J, CHEN X Y, et al. The Sharing Signal for Integrated Radar and Communication Based on FRFT. Journal of Signal Processing, 2012, 28(4): 487-494.) [12] HUANG T Y, SHLEZINGER N, XU X Y, et al. MAJoRCom: A Dual-Function Radar Communication System Using Index Modu-lation. IEEE Transactions on Signal Processing, 2020, 68: 3423-3438. [13] LIU F, ZHOU L F, MASOUROS C, et al. Toward Dual-Func-tional Radar-Communication Systems: Optimal Waveform Design. IEEE Transactions on Signal Processing, 2018, 66(16): 4264-4279. [14] LIU X, HUANG T Y, SHLEZINGER N, et al. Joint Transmit Beam-forming for Multiuser MIMO Communications and MIMO Radar. IEEE Transactions on Signal Processing, 2020, 68: 3929-3944. [15] LIU F, CUI Y H, MASOUROS C, et al. Integrated Sensing and Communications: Toward Dual-Functional Wireless Networks for 6G and Beyond. IEEE Journal on Selected Areas in Communica-tions, 2022, 40(6): 1728-1767. [16] ZHANG Z Q, XIAO Y, MA Z, et al. 6G Wireless Networks: Vision, Requirements, Architecture, and Key Technologies. IEEE Vehicular Technology Magazine, 2019, 14(3): 28-41. [17] CHENG X, ZHANG H T, ZHANG J N, et al. Intelligent Multi-modal Sensing-Communication Integration: Synesthesia of Machines. IEEE Communications Surveys and Tutorials, 2023. DOI: 10.1109/COMST.2023.3336917. [18] CHENG X, DUAN D L, GAO S J, et al. Integrated Sensing and Communications(ISAC) for Vehicular Communication Networks(VCN). IEEE Internet of Things Journal, 2022, 9(23): 23441-23451. [19] XU W H, GAO F F, TAO X M, et al. Computer Vision Aided mmWave Beam Alignment in V2X Communications. IEEE Tran-sactions on Wireless Communications, 2023, 22(4): 2699-2714. [20] ALI A, GONZÁLEZ-PRELCIC N, GHOSH A. Passive Radar at the Roadside Unit to Configure Millimeter Wave Vehicle-to-In-frastruc-ture Links. IEEE Transactions on Vehicular Technology, 2020, 69(12): 14903-14917. [21] JIANG S F, CHARAN G, ALKHATEEB A.LiDAR Aided Future Beam Prediction in Real-World Millimeter Wave V2I Communi-cations. IEEE Wireless Communications Letters, 2022, 12(2): 212-216. [22] 王洋,杨闯,彭木根.车联网场景下的视觉辅助太赫兹多用户波束跟踪.移动通信, 2023, 47(9): 89-95. (WANG Y, YANG C, PENG M G.Vision-Aided Terahertz Multi-user Beam Tracking in V2X. Mobile Communications, 2023, 47(9): 89-95.) [23] CHARAN G, ALRABEIAH M, ALKHATEEB A.Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Han-doff. IEEE Transactions on Vehicular Technology, 2021, 70(10): 10193-10208. [24] JIANG S F, ALKHATEEB A.Sensing Aided OTFS Channel Esti-mation for Massive MIMO Systems[C/OL]. [2023-09-09].https://arxiv.org/pdf/2209.11321.pdf. [25] XU W H, GAO F F, ZHANG J H, et al. Deep Learning Based Channel Covariance Matrix Estimation with User Location and Scene Images. IEEE Transactions on Communications, 2021, 69(12): 8145-8158. [26] XU W H, GAO F F, ZHANG Y, et al. Multi-user Matching and Resource Allocation in Vision Aided Communications. IEEE Tran-sactions on Communications, 2023, 71(8): 4528-4543. [27] FAN Y J, GAO S J, DUAN D L, et al. Radar Integrated MIMO Communications for Multi-hop V2V Networking. IEEE Wireless Communications Letters, 2022, 12(2): 307-311. [28] WIRGES S, FISCHER T, STILLER C, et al. Object Detection and Classification in Occupancy Grid Maps Using Deep Convo-lutional Networks // Proc of the 21st International Conference on Intelligent Transportation Systems. Washington, USA: IEEE, 2018: 3530-3535. [29] DOU J, XUE J R, FANG J W.SEG-VoxelNet for 3D Vehicle Detection from RGB and LiDAR Data // Proc of the International Conference on Robotics and Automation. Washington, USA: IEEE, 2019: 4362-4368. [30] STEYER S, LENK C, KELLNER D, et al. Grid-Based Object Tracking with Nonlinear Dynamic State and Shape Estimation. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(7): 2874-2893. [31] GU S, ZHANG Y G, TANG J H, et al. Integrating Dense LiDAR-Camera Road Detection Maps by a Multi-modal CRF Model. IEEE Transactions on Vehicular Technology, 2019, 68(12): 11635-11645. [32] BEELI G, ESSLEN M, JÄNCKE L. When Coloured Sounds Taste Sweet. Nature, 2005, 434: 38. [33] MATTINGLEY J B, RICH A N, YELLAND G, et al. Unconscious Priming Eliminates Automatic Binding of Colour and Alphanumeric Form in Synaesthesia. Nature, 2001, 410: 580-582. [34] RICH A N, MATTINGLEY J B. Anomalous Perception in Synaes-thesia: A Cognitive Neuroscience Perspective. Nature Reviews Neu-roscience, 2002, 3(1): 43-52. [35] FRANGEUL L, POUCHELON G, TELLEY L, et al. A Cross-Mo-dal Genetic Framework for the Development and Plasticity of Sen-sory Pathways. Nature, 2016, 538: 96-98. [36] NGUYEN D C, DING M, PATHIRANA P N, et al. 6G Internet of Things: A Comprehensive Survey. IEEE Internet of Things Jour-nal, 2022, 9(1): 359-383. [37] STAHLBUHK T, SHRADER B, MODIANO E.Learning Algori-thms for Scheduling in Wireless Networks with Unknown Channel Statistics. Ad Hoc Networks, 2019, 85: 131-144. [38] GOODFELLOW I, BENGIO Y, COURVILLE A. Deep Learning. Cambridge, USA: MIT Press, 2017. [39] ALKHATEEB A. DeepMIMO: A Generic Deep Learning Dataset for Millimeter Wave and Massive MIMO Applications[C/OL]. [2023-09-09].https://arxiv.org/abs/1902.06435. [40] MANIVASAGAM S, WANG S L, WONG K, et al. LiDARsim: Rea-listic LiDAR Simulation by Leveraging the Real World // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Recog-nition. Washington, USA: IEEE, 2020: 11164-11173. [41] XU R S, XIANG H, XIA X, et al. OPV2V: An Open Benchmark Dataset and Fusion Pipeline for Perception with Vehicle-to-Vehicle Communication // Proc of the International Conference on Robotics and Automation. Washington, USA: IEEE, 2022: 2583-2589. [42] BLANDINO S. Dataset of Channels and Received IEEE 802.11ay Signals for Sensing Applications in the 60GHz Band[DB/OL].[2023-09-09]. https://doi.org/10.18434/mds2-2417. [43] KLAUTAU A, GONZÁLEZ-PRELCIC N, HEATH R W, et al. LI-DAR Data for Deep Learning-Based mmWave Beam-Selection. IEEE Wireless Communications Letters, 2019, 8(3): 909-912. [44] KLAUTAU A, BATISTA P, GONZÁLEZ-PRELCIC N, et al. 5G MIMO Data for Machine Learning: Application to Beam-Selection Using Deep Learning // Proc of the Information Theory and Appli-cations Workshop. Washington, USA: IEEE, 2018. DOI: 10.1109/ITA.2018.8503086. [45] REMCOM. Wireless InSite[EB/OL].[2023-09-09]. https://www.remcom.com/wireless-insite-em-propagation-software. [46] ALRABEIAH M, HREDZAK A, LIU Z H, ViWi: A Deep Lear-ning Dataset Framework for Vision-Aided Wireless Communi-cations // Proc of the IEEE 91st Vehicular Technology Conference. Washington, USA: IEEE. DOI: 10.1109/VTC2020-Spring48590.2020.9128579. [47] CHENG X, HUANG Z W, BAI L, et al. M3SC: A Generic Data-set for Mixed Multi-modal(MMM) Sensing and Communication Integration. China Communications, 2023, 20(11): 13-29. [48] REMCOM. WaveFarer[EB/OL].[2023-09-09]. https://www.remcom.com/wavefarer-automotive-radar-software. [49] SHAH S, DEY D, LOVETT C, et al. AirSim: High-Fidelity Vi-sual and Physical Simulation for Autonomous Vehicles // HU-TTER M, SIEGWART R, eds. Field and Service Robotics(Re-sults of the 11th International Conference). Berlin, Germany: Springer, 2018: 621-635. [50] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: Op-timal Speed and Accuracy of Object Detection[C/OL].[2023-09-09]. https://arxiv.org/abs/2004.10934. [51] HUANG Z W, CHENG X, YIN X F.A General 3D Non-stationary 6G Channel Model with Time-Space Consistency. IEEE Transac-tions on Communications, 2022, 70(5): 3436-3450. [52] CHENG X, HUANG Z W, BAI L.Channel Nonstationarity and Consistency for Beyond 5G and 6G: A Survey. IEEE Communica-tions Surveys and Tutorials, 2022, 24(3): 1634-1669. [53] HUANG Z W, BAI L, SUN M R, et al. A Mixed-Bouncing Based Non-Stationarity and Consistency 6G V2V Channel Model with Continuously Arbitrary Trajectory. IEEE Transactions on Wireless Communications, 2023. DOI: 10.1109/TWC.2023.3293024. [54] HUANG Z W, BAI L, CHENG X, et al. A Non-stationary 6G V2V Channel Model with Continuously Arbitrary Trajectory. IEEE Transactions on Vehicular Technology, 2023, 72(1): 4-19. [55] ALRABEIAH M, HREDZAK A, ALKHATEEB A.Millimeter Wave Base Stations with Cameras: Vision-Aided Beam and Blockage Pre-diction // Proc of the IEEE 91st Vehicular Technology Confe-rence. Washington, USA: IEEE, 2020. DOI: 10.1109/VTC2020-Spring48590.2020.9129369. [56] GU J, COLLINS L, ROY D, et al. Meta-Learning for Image-Guided Millimeter-Wave Beam Selection in Unseen Environments // Proc of the IEEE International Conference on Acoustics, Speech and Signal Processing. Washington, USA: IEEE, 2023. DOI: 10.1109/ICASSP49357.2023.10096315. [57] DIAS M, KLAUTAU A, GONZÁLEZ-PRELCIC N, et al. Position and LIDAR-Aided mmWave Beam Selection Using Deep Learning // Proc of the IEEE 20th International Workshop on Signal Pro-cessing Advances in Wireless Communications. Washington, USA: IEEE, 2019. DOI: 10.1109/SPAWC.2019.8815569. [58] ZHENG Y D, CHEN S Z, ZHAO R.A Deep Learning-Based mmWave Beam Selection Framework by Using LiDAR Data // Proc of the 33rd Chinese Control and Decision Conference. Wa-shington, USA: IEEE,2021: 915-920. [59] DEMIRHAN U, ALKHATEEB A.Radar Aided 6G Beam Pre-diction: Deep Learning Algorithms and Real-World Demonstration // Proc of the IEEE Wireless Communications and Networking Conference. Washington, USA: IEEE, 2022: 2655-2660. [60] DEMIRHAN U, ALKHATEEB A.Radar Aided Proactive Blockage Prediction in Real-World Millimeter Wave Systems // Proc of the IEEE International Conference on Communications. Washington, USA: IEEE, 2022: 4547-4552. [61] SALEHI B, GU J, ROY D, et al. FLASH: Federated Learning for Automated Selection of High-Band mmWave Sectors // Proc of the IEEE Conference on Computer Communications. Washington, USA: IEEE, 2022: 1719-1728. [62] SALEHI B, REUS-MUNS G, ROY D, et al. Deep Learning on Multimodal Sensor Data at the Wireless Edge for Vehicular Net-work. IEEE Transactions on Vehicular Technology, 2022, 71(7): 7639-7655. [63] YANG Y W, GAO F F, TAO X M, et al. Environment Semantics Aided Wireless Communications: A Case Study of mmWave Beam Prediction and Blockage Prediction. IEEE Journal on Selected Areas in Communications, 2023, 41(7): 2025-2040. [64] LIU F, MASOUROS C.A Tutorial on Joint Radar and Communi-cation Transmission for Vehicular Networks-Part III: Predictive Beamforming without State Models. IEEE Communications Letters, 2021, 25(2): 332-336. [65] LIU F, YUAN W J, MASOUROS C, et al. Radar-Assisted Predictive Beamforming for Vehicular Links: Communication Served by Sensing. IEEE Transactions on Wireless Communications, 2020, 19(11): 7704-7719. [66] GAO S J, DONG Y, CHEN C, et al. Hierarchical Beam Sele-ction in mmWave Multiuser MIMO Systems with One-Bit Analog Phase Shifters // Proc of the 8th IEEE International Conference on Wire-less Communications and Signal Processing. Washington, USA: IEEE, 2016. DOI: 10.1109/WCSP.2016.7752457. [67] GAO S J, CHENG X, YANG L Q.Estimating Doubly-Selective Channels for Hybrid mmWave Massive MIMO Systems: A Doubly-Sparse Approach. IEEE Transactions on Wireless Communications, 2020, 19(9): 5703-5715. [68] GAO S J, CHENG X, FANG L Y, et al. Model Enhanced Lear-ning Based Detectors(Me-LeaD) for Wideband Multi-user 1-bit mmWave Communications. IEEE Transactions on Wireless Commu-nications, 2021, 20(7): 4646-4656. [69] FAN Y J, GAO S J, CHENG X, et al. Wideband Generalized Beam-space Modulation(wGBM) for mmWave Massive MIMO Systems over Doubly-Selective Channels. IEEE Transactions on Vehicular Technology, 2021, 70(7): 6869-6880. [70] SHAHAM S, DING M, KOKSHOORN M, et al. Fast Channel Esti-mation and Beam Tracking for Millimeter Wave Vehicular Commu-nications. IEEE Access, 2019, 7: 141104-141118. [71] MU J S, GONG Y, ZHANG F P, et al. Integrated Sensing and Communication-Enabled Predictive Beamforming with Deep Lear-ning in Vehicular Networks. IEEE Communications Letters, 2021, 25(10): 3301-3304. [72] ZHANG H T, GAO S J, CHENG X, et al. Integrated Sensing and Communications towards Proactive Beamforming in mmWave V2I via Multi-modal Feature Fusion(MMFF)[C/OL].[2023-09-09]. https://arxiv.org/pdf/2310.02561.pdf. [73] ZHU D L, CHOI J, CHENG Q, et al. High-Resolution Angle Tra-cking for Mobile Wideband Millimeter-Wave Systems with Antenna Array Calibration. IEEE Transactions on Wireless Communica-tions, 2018, 17(11): 7173-7189. [74] MUNDLAMURI R, GANGULA R, THOMAS C K, et al. Sensing Aided Channel Estimation in Wideband Millimeter-Wave MIMO Sys-tems[C/OL].[2023-09-09]. https://arxiv.org/pdf/2302.02065v1.pdf. [75] MA W Y, QI C H, ZHANG Z C, et al. Sparse Channel Estimation and Hybrid Precoding Using Deep Learning for Millimeter Wave Massive MIMO. IEEE Transactions on Communications, 2020, 68(5): 2838-2849. [76] WEI Y, ZHAO M M, ZHAO M J, et al. An AMP-Based Network with Deep Residual Learning for mmWave Beamspace Channel Esti-mation. IEEE Wireless Communications Letters, 2019, 8(4): 1289-1292. [77] LIU S C, HUANG X.Sparsity-Aware Channel Estimation for mm-Wave Massive MIMO: A Deep CNN-Based Approach. China Co-mmunications, 2021, 18(6): 162-171. [78] WEI X H, HU C, DAI L L.Deep Learning for Beamspace Cha-nnel Estimation in Millimeter-Wave Massive MIMO Systems. IEEE Transactions on Communications, 2021, 69(1): 182-193. [79] ALKHATEEB A, EL AYACH O, LEUS G, et al. Channel Esti-mation and Hybrid Precoding for Millimeter Wave Cellular Sys-tems. IEEE Journal of Selected Topics in Signal Processing, 2014, 8(5): 831-846. [80] DONOHO D L, MALEKI A, MONTANARI A.Message Passing Algo-rithms for Compressed Sensing: I. Motivation and Construction // Proc of the IEEE Information Theory Workshop on Information Theory. Washington, USA: IEEE, 2010. DOI: 10.1109/ITWKSPS.2010.5503193. [81] VAN DE BEEK J J, EDFORS O, SANDELL M, et al. On Cha-nnel Estimation in OFDM Systems // Proc of the IEEE 45th Vehi-cular Technology Conference. Washington, USA: IEEE, 1995: 815-819. [82] RAHMAN M L, ZHANG J A, HUANG X J, et al. Framework for a Perceptive Mobile Network Using Joint Communication and Radar Sensing. IEEE Transactions on Aerospace and Electronic Systems, 2019, 56(3): 1926-1941. [83] ZHENG X H, LI Y R, DUAN D L, et al. Multivehicle Multisensor Occupancy Grid Maps(MVMS-OGM) for Autonomous Driving. IEEE Internet of Things Journal, 2022, 9(22): 22944-22957. [84] ZHANG J W, ZHANG M, FANG Z C, et al. RVDET: Feature-Level Fusion of Radar and Camera for Object Detection // Proc of the IEEE International Intelligent Transportation Systems Confe-rence. Washington, USA: IEEE, 2021: 2822-2828. [85] ASVADI A, GARROTE L, PREMEBIDA C, et al. Multimodal Ve-hicle Detection: Fusing 3D-LIDAR and Color Camera Data. Pattern Recognition Letters, 2018, 115: 20-29. [86] PANG S, MORRIS D, RADHA H.CLOCs: Camera-LiDAR Ob-ject Candidates Fusion for 3D Object Detection // Proc of the IEEE/RSJ International Conference on Intelligent Robots and Systems. Washington, USA: IEEE, 2020: 10386-10393. [87] ZHENG X H, LI S J, LI Y K, et al. Confidence Evaluation for Machine Learning Schemes in Vehicular Sensor Networks. IEEE Transactions on Wireless Communications, 2023, 22(4): 2833-2846. [88] CHEN X Z, MA H M, WAN J, et al. Multi-view 3D Object De-tection Network for Autonomous Driving // Proc of the IEEE Con-ference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2017: 6526-6534. [89] QI C R, LIU W, WU C X, et al. Frustum Pointnets for 3D Object Detection from RGB-D Data // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Washington, USA: IEEE, 2018: 918-927. [90] WANG Z X, JIA K.Frustum ConvNet: Sliding Frustums to Aggre-gate Local Point-Wise Features for Amodal 3D Object Detection // Proc of the IEEE/RSJ International Conference on Intelligent Ro-bots and Systems. Washington, USA: IEEE, 2019: 1742-1749. [91] YANG Z T, SUN Y N, LIU S, et al. IPOD: Intensive Point-Based Object Detector for Point Cloud[C/OL].[2023-09-09]. https://arxiv.org/abs/1812.05276. [92] LI H, NASHASHIBI F.Multi-vehicle Cooperative Perception and Augmented Reality for Driver Assistance: A Possibility to 'See' through Front Vehicle // Proc of the 14th International IEEE Con-ference on Intelligent Transportation Systems. Washington, USA: IEEE, 2011: 242-247. [93] KIM S W, QIN B X, CHONG Z J,et al. Multivehicle Cooperative Driving Using Cooperative Perception: Design and Experimental Validation. IEEE Transactions on Intelligent Transportation Sys-tems, 2015, 16(2): 663-680. [94] CHEN Q, TANG S H, YANG Q, et al. Cooper: Cooperative Per-ception for Connected Autonomous Vehicles Based on 3D Point Clouds // Proc of the IEEE 39th International Conference on Dis-tributed Computing Systems. Washington, USA: IEEE, 2019: 514-524. [95] QIU H, HUANG P H, ASAVISANU N, et al. AutoCast: Scalable Infrastructure-Less Cooperative Perception for Distributed Colla-borative Driving[C/OL].[2023-09-09]. https://arxiv.org/pdf/2112.14947.pdf. [96] CHEN Q, MA X, TANG S H, et al. F-cooper: Feature Based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds // Proc of the 4th ACM/IEEE Sym-posium on Edge Computing. Washington, USA: IEEE, 2019: 88-100. [97] GUO J D, CARRILLO D, CHEN Q, et al. Slim-FCP: Light-Weight-Feature-Based Cooperative Perception for Connected Automated Vehicles. IEEE Internet of Things Journal, 2022, 9(17): 15630-15638. [98] HU J, SHEN L, SUN G.Squeeze-and-Excitation Networks // Proc of the IEEE/CVF Conference on Computer Vision and Pattern Re-cognition. Washington, USA: IEEE, 2018: 7132-7141. [99] XIAO Z Y, MO Z B, JIANG K, et al. Multimedia Fusion at Semantic Level in Vehicle Cooperactive Perception // Proc of the IEEE International Conference on Multimedia and Expo Work-shops. Washington, USA: IEEE, 2018. DOI: 10.1109/ICMEW.2018.8551565. [100] CHO H, SEO Y W, KUMAR B V K V, et al. A Multi-sensor Fusion System for Moving Object Detection and Tracking in Urban Driving Environments // Proc of the IEEE International Confe-rence on Robotics and Automation. Washington, USA: IEEE, 2014: 1836-1843. [101] WANG T H, MANIVASAGAM S, LIANG M, et al. V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Predi-ction // Proc of the 16th European Conference on Computer Vi-sion. Berlin, Germany: Springer, 2020: 605-621.
2023, Vol. 36 
